Modular Back-Propagation Neural Networks For Large Domain Pattern Classification

A significant problem associated with application of the Back Propagation learning paradigm for pattern classification is the lack of high accuracy in generalization when the domain is large. In this paper we describe a multiple neural network system, which uses two self-organizing neural networks that work as teaching data filters (feature extractors), producing information that is used to train a generalization neural network. We find that the modular neural network system described in this paper learns fast and generalizes very efficiently. The technique was successfully applied to the selection of control rules for a Traveling Salesman Problem (TSP) heuristic, thus making the TSP heuristic adaptive to the input problem instance. In these experiments, the multiple neural networks system was trained with a collection of 50 problems and achieved 100% accuracy in identifying the best control rules for the problems in the training set. These networks correctly classified 47 of 50 test problems not contained in the training set. In parallel experiments with single generalizing neural networks (with differing numbers of layers and topologies), generalization accuracy rates were in the 10-20% range. The high accuracy in the multiple network system described in this paper is especially noteworthy because, despite the importance of the problem in Operations Research, there was only a very limited body of knowledge upon which to base the control rule selection decision until this study was carried out. In addition, the results shown here should generalize to many applications employing back propagation for large domain data classification.